Some links will not be active if there is no internet connection.
BEWARE: this application can only run on anyflo version 16.2 jully 2014.
A Dynamic Analysis of the
fft audio signal (or its derivative) provides real-time, to each image, an M1 parameter list that is passed as input (pattern) of a
neural network another
L1 (law) parameters that are interpreted in terms of dynamic volumes and textures.
A list
M2of maximum modules fft increased input (pattern) of a second neural network which provides the output, another list
L1 (law) parameters that are interpreted in terms of succession of sequences.
En fait c´est l´In fact it is the
envelope de la fft of the fft being treate.
To assure processing in real a set of Image files, read at program startup time are interpolated to provide continuous transitions.
Networks are dynamically trained by algorithm
back-propagation of the error, its entries being updated periodically update the audio signal. It follows an animation leaning back permanently on music.
Different
effects can be programmedeffets.
Note:
Message "Must press w key to start audio" appears.
HEL
Help.
EXP
HEL
Help.
AUD
HEL
RING
AMP
FFT
STOP
HEL
Help.
RING
Change
freq (frequency) and
dur (duration).
AMP
Change
audio and
amplitude.
FFT
Change
audio and
amplitude.
STOP
Stops sound.
NET
HEL
BUILD
LEARN
MATRIX
HEL
Help.
BUILD
Building a neural network.
Change
couples=number of training couples.
Change
hide=number of hidden layers.
LEARN
Learning the neural network, produce sound.
MATRIX
Displays the matrix of synaptic weights.
INPUT
HEL
Help.
AUD
Audio.
CAM
Camera: The window is shifted to the right, leaving room for the window it took
WebCam.exe launch earlier stages, webcam well as a camp whose image properly resized,
has been brought on the first, otherwise. the program crashes ...
Change scales:
liminance rgb: luminance (1D) or RGB (3D).
dim cam: size of the camera image displayed..
secx secy: size of the captured image, also dimension of patterns and laws of network that will rebuild.
speed: rate of change patterns.
Then just click on AUTO to start the process.
OUTPUT
HEL
Aide.
INI: ini image
IMA
HEL
READ
DIRECTORY
BUILD
PIXEL
Ces différents modes peuvent être combinés.
HEL
Help.
READ
reads images: click on a directory name, hen on a name.bmp.
DIRECTORY
lit reads images folders: click on a directory name.
BUILD
builds random images, click on nb = number of images
and on complex = complexity;
PIXEL
builds images curves, click on nb = complexity;
CAM
Web cam captures an image and displays transparent texture (must be launched webcam.js and have launched a web cam and bringing its image (appropriately scaled) on the previous window, then click click here.
CAM2
As CAM2, but the image is captured
only when the transparency of the last layer is null.
NET1
Neural network transforming the fft of the audio sugnal.
Change couples (number of learning couples); and
hide (number of hidden layers).
HEL
Aide.
INI
Removes the network.
LEARN
Launches the learning process.
On peut
see
the input layer neurons (green), those of the ouptut layer (red)
as well as those of the hidden layers activate.
MATRIX
Displays the matrix of synaptic weights.
NET2
Neural network transforming lelory Memroy into values
passed to interaction image(1,Nb_Seq)vol law(m); in order
to determin the following sequence. Change:
couples (number of learning couples); and
hide (number of hidden layers).
HEL
Help.
INI
Supprime le réseau.
LEARN
Launches the learning process.
We can
see
the input layer neurons (green), those of the output layetr
(red) as well as those of the hidden layers activate.
On the right we can see the error curve (white).
MATRIX
displays the matrix of synaptic weights.
AUTO
Launches the process.
Click on reset change the refresh period of the network.
Click on ni change the of interpolated images interpolées, this may take some time.
HEL
Help.
SELF
Activates the
automapping (keyboard shortcut s).
SHOW
Displays the audio signal, the motifs and the laws of both networks
(even if : is active).
PREC
image precision sends a thresholding of the image.
LIGHT
Computes the color of light depending on the average fft
(keyboard shortcut l).
COL
Computes the color of volume depending on the average fft, change the scales coe COL and pow (a small value of pow improves contrasts.
VIEW
Variable point of view (keyboard shortcut v).
VOL
Volume trajectories (keyboard shortcut V).
SIN
Sinusoidal variationde of the texture increment.
net
Transforms (or not) the audio signal audio by the network Net1 (keyboard shortcut n).
DEB
Debug mode: prints some parameters for the launch(heyboard shortcut d).
DER
Does Der[2]=1 (sensor speed) (keyboard shortcut D).
CONT
Contour extraction.
MOVE
Fixed points of the grid (their boundaries) are moved according to the fft, change the scale coe MOVE (keyboard shortcut M).
BACK
The background color varies (keyboard short shortcut b).
NOISE
The forces applied to the vertices are noised of coe NOISE, available on SCALES VOL (keyboard shortcut Z).
NEXT
Following sequence.
ALEA
Random sequences.
LAW
Sequences determined by the maximum value of the memory Memory by the network Net2
(keyboard shortcut L).
AMP
Sequences determined by Module*max(number image).
(keyboard shortcut a).
CUR
sequence determined by law(Module,Curve) (keyboard shortcut k).
To change the curve Curve click on SCALES -> PARAM, change
np, then the curve.
TRA
Images are mixed by image(i1,i2)center(x,y)radius(rx,ry)transp(t1,t2)normal
TAB
The look up table is changed according to the values of the module stocked in the FILO stack Memory.
WAIT
Audio OFF/ON (ON defautl: must press w key to start audio.
SCALES
HEL
Help.
OUTPUT
dim: images.
dim: image size.
r,g,b: coefficients thresholds color if PREC is active.
AUD
audio: number of samples= size of the input
motifs = 2^audio. It is then necessary to click on AUTO.
envelope: envelope.
above: if validates audio var sort above (signal reduced to the maximum component).
derive: capture = speed.
min: minimum of the audio signal, allows ignore too low values.
maw: maximum of the audio signal, allows ignore too hight values.
amp: multiplier signal.
exp,adjust,rand:
signal adjustement.
per: refresh period motifs;
speed: textures speed variation.
coe audio=c:
if c >0: does coe audio=c.
if <0: choose m1 and m2 making coe audio=m1,m2
(automatic coefficient).
CAM
luminance rgb: luminance or RVG.
dim cam: image (power of 2).
secx secy: camera resolution.
resolution
VOL
texture: number of textures.
If CONT is validated:
pricision and smooth
stiffness,coe stiffness and viscosity of the springs..
max maximum textures increment (avoids swings).
viscosity: viscosity.
hide: number of hidden layers.
ni: determines the size of the image interpolation.
sin1,sin2: if SIN is va lidated changes the sinusoidalvariations of the textures increment.
PARAM
If href="#TAB">TAB is active, displays thelook up table curve.
If raw is activedisplay the name of the image file currently processed and the scale coe period.
The files must be in format raw.
If option CUR is active, displays np and the curve
Curve thar can be changed.
RESET
Restores original (afetr, for example, SELF or
PREC).
ILLUM
yes/no illum.
EXT
yes/no ext, change the scaleext.
INI
Restores the original parameters.
EXI
Exits.
: (colon) hides menus.
A: random suquences (otherwise linear) pour
m mini menu appears in the upper left with options:
A ALEA
b BACK
c COL
C CONT
D DER
d DEB
h help mini menu (keyboard shortcut).
i initialisation
l LIGHT
L LAW
m mini menu
M MOVE
n NET
N NEXT
p PREC
r RESET
s SELF
S SHOW
t TRA
T TAB
v VIEW
V VOL
Z NOISE
: cache les menus
+ incrments amp
- decrements amp
q EXI
Direct
Click on neuro_VJ_raw.js launching
neuro_VJ.func with option raw
which dynamically read images format raw of directory RAW.
Click on neuro_VJ_bmp.js launching
neuro_VJ.func with option
dir=ima/BMP which reads images format bmp of directory
BMP.
Click on webcam.js launching a shared memory driver.
Then launche a webcam, drag the resized image on the window of the driver
(lower left corner).
Then click on neuro_VJ_cam.js launching
neuro_VJ.func with options:
cam readin the captured image.
and dir=ima/BMP which reads images format bmp
of directory BMP.
Play music and act front of the camera.
Manual
Lancer neuro_VJ.js without option.
Click on INPUT then on AUD, making audio the input sensor.
Click on OUTPUT then on
IMA then on
DIRECTORY and
select folders>, this starts the process: all images of selected folders
are loaded and used as interpolated transparency textures.
Click on AUTO
Press : (colon) key hidding menus.
Press m key giving mini menu.
Pressh key giving mini menu help.
Press documented keys to make several effects.
Press : key gives menus.
Click on SCALES, then,
OUTPUT.
AUD.
VOL.
NET1.
NET2.
Activate NET1, NETWORK->LEARN,
and AUTO.
Note: if INPUT_AUD is validated, images must have been read.
The function
The function neuro_VJ.func is in the folder ima.
The sequences
If net is not validated the signal is directly interpreted
(the message DIRECT is displayed above).
Otherwise the signal is passed in entrance of the neural network Net1 which
gives as output the interpreted signal. It is necessary to have read the
images or to have specified an option dir.
The option dir=dir_1,dir_2,...,dir_n produces succession of sequences
according to various criteria:
random (keyboard shortcut A).
next (keyboard shortcut N).
law (keyboard shortcut L).
amplitude (keyboard shortcut a).
curve (keyboard shortcut k).
One can build as many as such directories as needed. But,
as the images are loaded in memory, the number of images they contain is limited
according to memory size.
In that case the option raw allows dynamically read as many
images as needed. For that it will be necessary to launch
bmp_raw.js.
Volumes
A grid is buildt with:
secx = secy = Nxy = dim(Capture)
memory(1) = numbers of summits not subjected to the dynamic (but directly modified by the fft.
memory(2) = coordinates of those points.
ver=memory(1)vol(num);n=dim(ver);p=memory(2)vol(num);
(in function func_AUTO)
memory("a")vol(num)=(average(Capture))^e;
aud=n: does Nb_AUDIO[2]=n (9 default).
aut: activates the automatic operation automatique.
cam: activates the camera.
cont=p: contour of precision p (0.025 default).
cur=0,0,0, x1,y1,0, x2,y2,0, ..., 1,1,0: Curve.
der: doest Derive[2]=1 (sensor speed).
dim=d: size of the images=2^d (7 default).
dir=ima/BMP: reads images of folder BMP.
dir0=dos: reads folders of folder dos.
env=n: does Envelope[2]=n (6 default).
1: gives an example of programmable option, see case 1: of
function option(w) reading two directories BMP1,BMP2. File
neuro_VJ_1.js uses such an option.
raw: only active images are loaded allowing
interpolations between an unlimited number of images with arbitrariy size (butin this case, option TAB does not work in real time).
The interpolation of textures is made by transparency with properties:
texture vol(1)=t1,t2,... and
transp texture vol(1)=tr1,tr2,...).
The array Transp = {(transp,signe,0)i of n 3D vectors 3D allows managing the succession of sequences.
texture=nb: number of textures.
Example:
quot;dir=ima/BMP1,ima/BMP2;
reads images of folders ima/CAH_1 and ima/SUZ_1
ni=n: number of intercalated images between 2 consecutive images (12 default).
:: hides menus.
a: random succession of images (otherwise linear).
b: activates the background color.
c: activates the colors.
l: activates the light.
p: activates limage precision.
q: inhibits the q key (avoiding existing).
s: activates the selfomapping.
t: activates transparency.
T: activates the look up table.
v: activates function VIEW.
V: activates function VOL.
Z: activates function NOISE with coefficient c.
Z: does WAIT OFF.
neuro_VJ.js: launch without option.
neuro_VJ_cam.js: option cam
neuro_VJ_cam2.js: option cam2
neuro_VJ_raw.js: option raw
Note:
If at least one option is precised (automatic operation) the option
EXI is inhibited, to exit must:
: retriving menus.
and click on EXI.
File jpg_bmp.js launches
file jpg_bmp.func allowing converting images
from ima/JPG/*.jpg of any size in ima/BMP/*.bmp of size 256.
File bmp_raw.js launches
file bmp_raw.func allowing converting images
from ima/BMP/*.bmp of size 256 in ima/RAW_2/*.raw.